70 research outputs found
The Grammar of Interactive Explanatory Model Analysis
The growing need for in-depth analysis of predictive models leads to a series
of new methods for explaining their local and global properties. Which of these
methods is the best? It turns out that this is an ill-posed question. One
cannot sufficiently explain a black-box machine learning model using a single
method that gives only one perspective. Isolated explanations are prone to
misunderstanding, which inevitably leads to wrong or simplistic reasoning. This
problem is known as the Rashomon effect and refers to diverse, even
contradictory interpretations of the same phenomenon. Surprisingly, the
majority of methods developed for explainable machine learning focus on a
single aspect of the model behavior. In contrast, we showcase the problem of
explainability as an interactive and sequential analysis of a model. This paper
presents how different Explanatory Model Analysis (EMA) methods complement each
other and why it is essential to juxtapose them together. The introduced
process of Interactive EMA (IEMA) derives from the algorithmic side of
explainable machine learning and aims to embrace ideas developed in cognitive
sciences. We formalize the grammar of IEMA to describe potential human-model
dialogues. IEMA is implemented in the human-centered framework that adopts
interactivity, customizability and automation as its main traits. Combined,
these methods enhance the responsible approach to predictive modeling.Comment: 17 pages, 10 figures, 3 table
archivist: An R Package for Managing, Recording and Restoring Data Analysis Results
Everything that exists in R is an object [Chambers2016]. This article
examines what would be possible if we kept copies of all R objects that have
ever been created. Not only objects but also their properties, meta-data,
relations with other objects and information about context in which they were
created.
We introduce archivist, an R package designed to improve the management of
results of data analysis. Key functionalities of this package include: (i)
management of local and remote repositories which contain R objects and their
meta-data (objects' properties and relations between them); (ii) archiving R
objects to repositories; (iii) sharing and retrieving objects (and it's
pedigree) by their unique hooks; (iv) searching for objects with specific
properties or relations to other objects; (v) verification of object's identity
and context of it's creation.
The presented archivist package extends, in a combination with packages such
as knitr and Sweave, the reproducible research paradigm by creating new ways to
retrieve and validate previously calculated objects. These new features give a
variety of opportunities such as: sharing R objects within reports or articles;
adding hooks to R objects in table or figure captions; interactive exploration
of object repositories; caching function calls with their results; retrieving
object's pedigree (information about how the object was created); automated
tracking of the performance of considered models, restoring R libraries to the
state in which object was archived.Comment: Submitted to JSS in 2015, conditionally accepte
How much should you ask? On the question structure in QA systems
Datasets that boosted state-of-the-art solutions for Question Answering (QA)
systems prove that it is possible to ask questions in natural language manner.
However, users are still used to query-like systems where they type in keywords
to search for answer. In this study we validate which parts of questions are
essential for obtaining valid answer. In order to conclude that, we take
advantage of LIME - a framework that explains prediction by local
approximation. We find that grammar and natural language is disregarded by QA.
State-of-the-art model can answer properly even if 'asked' only with a few
words with high coefficients calculated with LIME. According to our knowledge,
it is the first time that QA model is being explained by LIME.Comment: Accepted to Analyzing and interpreting neural networks for NLP
workshop at EMNLP 201
Adversarial Attacks and Defenses in Explainable Artificial Intelligence: A Survey
Explainable artificial intelligence (XAI) methods are portrayed as a remedy
for debugging and trusting statistical and deep learning models, as well as
interpreting their predictions. However, recent advances in adversarial machine
learning (AdvML) highlight the limitations and vulnerabilities of
state-of-the-art explanation methods, putting their security and
trustworthiness into question. The possibility of manipulating, fooling or
fairwashing evidence of the model's reasoning has detrimental consequences when
applied in high-stakes decision-making and knowledge discovery. This survey
provides a comprehensive overview of research concerning adversarial attacks on
explanations of machine learning models, as well as fairness metrics. We
introduce a unified notation and taxonomy of methods facilitating a common
ground for researchers and practitioners from the intersecting research fields
of AdvML and XAI. We discuss how to defend against attacks and design robust
interpretation methods. We contribute a list of existing insecurities in XAI
and outline the emerging research directions in adversarial XAI (AdvXAI).
Future work should address improving explanation methods and evaluation
protocols to take into account the reported safety issues.Comment: A shorter version of this paper was presented at the IJCAI 2023
Workshop on Explainable A
- …